Apparatus and method for providing workload distribution of threads among multiple compute units

ABSTRACT

In some examples, thermal aware optimization logic determines a characteristic (e.g., a workload or type) of a wavefront (e.g., multiple threads). For example, the characteristic indicates whether the wavefront is compute intensive, memory intensive, mixed, and/or another type of wavefront. The thermal aware optimization logic determines temperature information for one or more compute units (CUs) in one or more processing cores. The temperature information includes predictive thermal information indicating expected temperatures corresponding to the one or more CUs and historical thermal information indicating current or past thermal temperatures of at least a portion of a graphics processing unit (GPU). The logic selects the one or more compute units to process the plurality of threads based on the determined characteristic and the temperature information. The logic provides instructions to the selected subset of the plurality of CUs to execute the wavefront.

GOVERNMENT LICENSE RIGHTS

This invention was made with Government support under PathForwardProject with Lawrence Livermore National Security (Prime Contract No.DE-AC52-07NA27344, Subcontract No. B620717) awarded by the Department ofEnergy (DOE). The Government has certain rights in this invention.

BACKGROUND OF THE DISCLOSURE

Graphics processing units (GPUs) have emerged as a first-class computingplatform. The massive data parallelism of GPUs was first leveraged byhighly-structured parallel tasks such as matrix multiplications.However, GPUs have more recently found use across a broader range ofapplication domains such as deep learning, graph analytics, dataanalytics, computer-aided-design, oil and gas exploration, medicalimaging, cloud computing, supercomputing, and computational finance.

Each GPU includes multiple compute units to execute wavefronts.Wavefronts are sets of threads, which are different sequences ofprogrammable instructions. Traditionally, the GPU distributes threads tocompute units on a first come, first serve basis. In other words, when awork queue buffer obtains a set of threads, the GPU assigns theunexecuted threads to any available, idle compute unit. However, certaincompute units have associated higher thermal costs (e.g., proximity tohigh temperature regions) for executing the threads than others.Eventually, this may cause the GPU to slow down and/or overheat. Assuch, there exists a need for improved methods and apparatus in order toaddress one or more of the above-noted drawbacks.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments will be more readily understood in view of the followingdescription when accompanied by the below figures and wherein likereference numerals represent like elements, wherein:

FIG. 1 is a schematic block diagram illustrating an example of anapparatus for providing workload distribution of threads among multiplecompute units in accordance with one variation set forth in thedisclosure;

FIG. 2 is an exemplary representation of a second processor, such as agraphics processing unit, in accordance with one variation set forth inthe disclosure;

FIG. 3 an exemplary representation of processing units and compute unitsin accordance with one variation set forth in the disclosure;

FIG. 4 an exemplary representation of compute units in accordance withone variation set forth in the disclosure;

FIG. 5 is a schematic block diagram illustrating an example of a thermalaware optimization logic for distributing threads among multiple computeunits in accordance with one variation set forth in the disclosure;

FIG. 6 is a flowchart illustrating a method for providing workloaddistribution of threads among multiple compute units in accordance withone variation set forth in the disclosure;

FIG. 7 illustrates an example of a heat map of a graphics processingunit in accordance with one variation set forth in the disclosure; and

FIG. 8 is an exemplary representation of one or more layered graphicsprocessing units in accordance with one variation set forth in thedisclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

In some examples, methods and apparatuses provide workload distributionof threads among multiple compute units. For example, since temperatureis a function of power density and not power alone, scheduling workappropriately on the CUs becomes an important problem in order toextract the most performance out of the GPU. Thus, the method andapparatus distribute threads (e.g., wavefronts) to selected computeunits based on thermal feedback of the graphics processing unit (GPU),locations of heat sources (e.g., components) of the GPU, and/orcharacteristics of the wavefront. By selecting CUs based on thermalinformation (e.g., distributing the wavefronts across differentlocations within the GPUs), the method and apparatus reduces the maximumtemperature of the GPU and/or improves performance of the GPU byreducing the effect of thermal throttling (e.g., throttling of the GPU'sclock frequencies).

In some variations, thermal aware optimization logic determines acharacteristic (e.g., a workload or type) of a wavefront (e.g., multiplethreads). For example, the characteristic indicates whether thewavefront is compute intensive, memory intensive, mixed, and/or anothertype of wavefront. The thermal aware optimization logic determinestemperature information for one or more compute units (CUs) in one ormore processing cores. The temperature information includes predictivethermal information indicating expected temperatures corresponding tothe one or more CUs and historical thermal information indicatingcurrent or past thermal temperatures of at least a portion of a graphicsprocessing unit (GPU). The logic selects the one or more compute unitsto process the plurality of threads based on the determinedcharacteristic and the temperature information. The logic providesinstructions to the selected subset of the plurality of CUs to executethe wavefront.

In some instances, the logic determines characteristics for multiplewavefronts currently being executed by the CUs. The logic generates thepredictive thermal information based on the characteristics for thewavefronts being executed by the CUs. In some examples, the portion ofthe GPU that the logic has historical thermal information for includesat least one non-CU circuitry, such as a memory controller, heat source,and/or other non-CU components within the GPU. In some variations, thelogic obtains temperature readings from multiple temperature sensorsoperatively coupled to different regions and/or components of the GPU.The logic generates historical thermal information based on thetemperature readings from the temperature sensors.

In some variations, a processor (a first GPU) includes the logic.Further, the logic determines characteristics for wavefronts beingexecuted by CUs on another (a second GPU) processor. The processors arestacked on top of each other. The logic generates predictive thermalinformation based on the characteristics of the wavefronts beingexecuted by the CUs on the second GPU. In some instances, the second GPUincludes at least one non-CU circuitry. The logic generates predictivethermal information based on a location of the non-CU circuitry. In someexamples, the second GPU includes temperature sensors. The logic obtainstemperature readings from the temperature sensors on the second GPU. Thelogic generates historical thermal information based on the temperaturereadings from the temperature sensors.

In some variations, the logic determines whether the wavefront is amemory intensive wavefront, a computational intensive wavefront, or amixed (e.g., memory and computational intensive) wavefront. In someexamples, the logic selects the subset of CUs based on whether thewavefront is memory intensive, computational intensive, or mixed.

FIG. 1 illustrates one example of an apparatus 100 for providingworkload distribution of wavefronts (e.g., threads) among multiplecompute units. In some implementations, the apparatus 100 includes anytype of computing device suitable for implementing aspects ofembodiments of the disclosed subject matter. Examples of computingdevices include but are not limited to workstations, laptops, desktops,tablet computers, hand-held devices, display devices, media players,set-top boxes, televisions, game consoles, printers, servers, cloudcomputing platforms, integrated circuits and the like, all of which arecontemplated within the scope of FIG. 1, with reference to variouscomponents of the apparatus 100.

In some examples, the apparatus 100 includes a bus 112 that directlyand/or indirectly couples the following devices: a first processor(e.g., central processing unit (CPU)), one or more second processors(e.g., graphics processing unit (GPU), general purpose GPU (GPGPU),and/or accelerated processing unit (APU)), and apparatus or systemmemory 120. Any number of additional components, different components,and/or combinations of components is also included in the apparatus 100.

The bus 112 represents one or more busses (such as, for example, anaddress bus, data bus, or combination thereof). Similarly, in someimplementations, the apparatus 100 includes a number of first processors102 and/or a number of memory components 640. Further, solely for thesake of brevity, four second processors 104, 106, 108, 110 are shown inFIG. 1. However, in some variations, the apparatus 100 includes anynumber of second processors (e.g., GPUs). For example, in one variation,the apparatus 100 includes one GPU. In other variations, the apparatus100 includes ten or fifteen GPUs. Additionally, any number of thesecomponents, or combinations thereof, is distributed and/or duplicatedacross a number of computing devices.

In some examples, each of the second processors 104, 106, 108, 110includes processing units 114, thermal aware optimization logic 116, andcache memory 118 (e.g., a work queue buffer). The thermal awareoptimization logic (logic) 116 obtains wavefronts (e.g., a set ofthreads) and thermal information. Then, based on the thermal informationand a characteristic of the wavefront, the logic 116 selects one or morecompute units (CUs) to execute the wavefront, and provides instructionsto the selected CUs to execute the wavefront. The operation of theprocessing units 114, thermal aware optimization logic 116, and cachememory 118 are described in further detail below.

In some examples, the system memory 120 includes computer-executableinstructions 122 that when executed cause the first processor 102 andsecond processors 104, 106, 108, 110 to implement aspects of embodimentsof apparatus components discussed herein and/or to perform aspects ofembodiments of methods and procedures discussed herein. The memory 120may be RAM, ROM, or any suitable memory. The illustrative apparatus 100,including the thermal aware optimization logic 116, is not intended tosuggest any limitation as to the scope of use or functionality ofembodiments of the present disclosure.

FIG. 2 illustrates an example of a second processor that providesworkload distribution of wavefronts (e.g., threads) among multiplecompute units. In particular, FIG. 2 shows one of the second processors104, 106, 108, 110. Below, the components of the second processor 104are described solely for brevity, and the other second processors ofapparatus 100, including second processors not shown in FIG. 1, canfunction similarly and/or include components similar to second processor104.

The second processor 104 includes cache memory 118 (e.g., a work queuebuffer), the logic 116, non-CU circuitry 220, non-CU circuitry 222, andprocessing units 114. The cache memory 118, logic 116, and the non-CUcircuitry 220 and 222 will be described in further detail below. FIG. 3illustrates an example of the processing units 114. The processing units114 includes one or more temperature sensors 214, CUs 206, 208, 210,212, and a bus 302. The bus 302 directly and/or indirectly couples thetemperature sensors 214 and CUs 206, 208, 210, 212. Additionally, and/oralternatively, the CUs 206, 208, 210, 212 use the bus 302 to obtaininformation 204, 218, 224 and/or provide information 218, 224, 216 toone or more components within the second processor 104, GPU 104, and/orother devices within the apparatus 100. Any number of additionalcomponents, different components, and/or combinations of components canalso be included in the processing units 114.

In particular, the processing units 114 includes sets of one or more CUs206, 208, 210, 212, temperature sensors 214, and a bus 302. Each of thesets of CUs 206, 208, 210, and 212 includes one or more CUs. In someinstances, a shader engine includes a group or set of CUs (e.g., 206,208, 210, 212). The GPU includes at least one group of shader engines(e.g., the group of the CUs 206, 208, 210, 212). In some variations, thesets of CUs (e.g., a shader engine) is referred to as a processor coreor GPU core. In some examples, the processing units 114 includes 64 CUs.However, the GPU 104 can include any number of CUs within the processingunits 114. Additionally, and/or alternatively, in some instances, theGPU 104 includes more than one processing units 114, and each of theprocessing units 114 includes a number of CUs.

Further, as shown in FIGS. 2 and 3, the processing units 114 includesone or more temperature sensors 214. Each temperature sensor 214 detectsand/or provides temperature readings or feedback 216 to the logic 116.The temperature sensor 214 can be any sensor, such as an on-dietemperature sensor, that detects and/or provides temperature feedback216 to the logic 116. In some examples, each CU within the sets of CUs206, 208, 210, 212 includes and/or is operatively coupled to one or moretemperature sensors 214. In other instances, some CUs within the sets ofCUs 206, 208, 210, 212 include a temperature sensor 214 and some CUs donot include a temperature sensor 214. Additionally, and/oralternatively, the processing units 114 includes multiple temperaturesensors 214 that are not operatively coupled to or included with the CUs(e.g., in between the CUs).

Also, the GPU 104 includes temperature sensors 214 that are not withinthe processing units 114. For example, the non-CU circuitry 220 and 222includes and/or is operatively coupled to a temperature sensor 214.Additionally, and/or alternatively, the GPU 104 also includes othertemperature sensors 214 (not shown) that are not within the processingunits 114 and/or the non-CU circuitry 220 and 222.

FIG. 4 illustrates an example of the components of a CU shown in FIGS. 2and 3. For example, compute units of the GPU are basic computationalblocks of the GPU 104, and there are typically 8 to 64 CUs in a GPU 104.Each CU, such as a CU frm the set of CUs 206, includes multipleSingle-Instruction-Multiple-Data (SIMD) units 408, each SIMD unit 408has multiple (e.g., 16) lanes of execution. Threads are scheduled toSIMD units 408 in a bundle called a wavefront (or warp), which istypically composed of 32 or 64 threads. These wavefronts execute thesame instruction with different data on a SIMD unit 408.

The memory resources of the GPU 104 are also arranged in a hierarchy.For example, each CU 206 has a private L1 data cache 406 and ascratchpad 410 that are shared across the SIMD units 408 within the CU206. Further, a large L2 cache 118 is shared across all CUs 206, 208,210, 212 in a GPU 104. Also, a global data store 412 is scratchpadmemory for the GPU 104, and is used by GPU applications to improveperformance. The non-CU circuitry 220 is any circuitry that is withinthe GPU 104 and is not a CU. Non-CU circuitry 220 will be described infurther detail below.

Below, the set of compute unit(s) 206 will be described solely forbrevity, and the other compute units 208, 210, 212 of apparatus 100,including other compute units not shown in FIG. 2, can functionsimilarly and/or include components similar to the compute unit(s) 206.Further, FIGS. 2, 3, and 4 will be described together below, and willreference the apparatus 100 of FIG. 1.

Referring to FIG. 2, in operation, the work queue buffer 118 (e.g., awork queue buffer stored in cache memory, such as L2 or L3 cache)obtains and stores wavefronts 200. For example, the CPU 102 and/oranother entity executes instructions and/or provides wavefronts 200 tothe work queue buffer 118. Wavefronts are a set (e.g., multiple) ofthreads, and each thread includes one or more programmable instructions.

The work queue buffer 118 stores the one or more wavefronts from the CPU102 and queues the wavefronts. The logic 116 obtains (e.g., receivesand/or retrieves) a wavefront from the work queue buffer 118. The logic116 determines characteristics of the wavefront and temperatureinformation for the GPU 104. Based on the determined characteristic andtemperature information, the logic 116 provides instructions to aselected CU or set of CUs (e.g., 206) to execute the wavefront. After,the logic 116 obtains another wavefront, and the process repeats. Theoperation of logic 116 will be described in further detail below.

The CUs 206, 208, 210, 212 from the processing units 114 execute thewavefronts and provides information to CUs within the processing units114, components of the GPU 104, other devices within the apparatus 100,and/or other apparatuses that are in communication with (e.g.,wirelessly and/or wired) the apparatus 100. The temperature sensor 214detects, determines, and/or provides temperature feedback 216 to thelogic 116.

The CUs 206, 208, 210, 212 use bus 302 to obtain and/or provideinformation 218, 224 to non-CU circuitry 220, 222. The non-CU circuitry220 and 222 is any component, entity, or circuitry within the GPU 104that is not a CU and thermally impacts the GPU 104. In other words, thenon-CU circuitry 220, 222 are heat sources and/or sinks within the GPU104 that impacts temperature of the GPU 104. In some examples, thenon-CU circuitry 220 and 222 are input/output units, memory controllers,crossbar bridges, through silicon vias (TSVs) and/or other circuitrywithin the GPU 104. In other examples, the non-CU circuitry 220, 222 isnot circuitry but a vent or other entity that permits air flow and/orcooling to the GPU 104. Solely for the sake of brevity, two non-CUcircuitry 220, 222 are shown in FIG. 2. However, in some variations, theapparatus 100 and/or the GPU 104 includes any number of non-CUcircuitry.

FIG. 5 illustrates an example of the thermal aware optimization logic116 shown in FIGS. 1 and 2. However, any suitable structure can beemployed. The logic 116 includes a characteristic unit 502, aneighboring circuit determination unit 504, a historical and currentimpact determination unit 506, and a granularity scheduler unit 508.Although these sub-units 502, 504, 506, and/or 508 are illustrated aschildren units subordinate of the parent logic 116, each sub-unit can beoperated as a separate unit from the logic 116, and other suitablecombinations of sub-units are contemplated to suit differentapplications.

In some examples, the logic 116 is a thermal aware hardware scheduler,thermal aware hardware dispatcher, and/or a thermal aware hardwareshader processor input (SPI) within a GPU, such as the GPU 104. However,in other examples, the logic 116 is any thermal aware wavefrontdistribution component that obtains and/or distributes wavefronts toselected CUs. In yet other examples, the logic 116 is any suitable logicconfiguration including, but not limited to, one or more state machines,one or more digital signal processors, as part of one or more CPUs,GPUs, APUs or processors that execute kernels, and/or other suitablestructure as desired. As used herein, the term “unit” refers to, be partof, or include an Application Specific Integrated Circuit (ASIC), anelectronic circuit, a processor or microprocessor (shared, dedicated, orgroup) and/or memory (shared, dedicated, or group) that executes one ormore software or firmware programs, a combinational logic circuit,and/or other suitable components that provide the describedfunctionality.

FIG. 6 illustrates an example of a method 600 for providing workloaddistribution of a wavefront among CUs. FIG. 6 will be described withreference to the apparatus 100 illustrated in FIGS. 1, 2, and 3 and thelogic 116 illustrated in FIG. 5. However, any suitable structure can beemployed.

In operation, at step 602, the logic 116 obtains a wavefront. Forexample, the characteristic unit 502 obtains information 202 includingone or more wavefronts from the work queue buffer 118. As mentionedpreviously, a wavefront is multiple or a set of threads.

At step 604, the characteristic unit 502 determines a characteristic(e.g., workload) of the wavefront. For example, the characteristic unit502 categories the obtained wavefront into one or more types ofwavefronts. The characteristic unit 502 provides information 514indicating the characteristic of the wavefront to the neighboringcircuit determination unit 504 and/or the granularity scheduler unit508. The types of wavefronts include, but are not limited to, a computeintensive wavefront, a memory intensive wavefront, and/or a mixed (e.g.,mixture of computational intensive and memory intensive) wavefront.

The logic 116 selects CUs to execute the wavefront based on thecharacteristic (e.g., type or workload) of the wavefront. For example,the logic 116 provides thermal aware distribution of wavefronts toreduce the temperature of the GPU 104 (e.g., on-die temperatures),prevent reduction in performance of the GPU 104 due to prematurethrottling of clock frequencies, and/or prevent the GPU 104 fromoverheating. The type or characteristic of the wavefront impacts thetemperature of the GPU 104 differently.

For example, a memory intensive wavefront has a lower thermal impactwhen compared to a compute intensive wavefront. A mixed intensivewavefront has a higher thermal impact than a memory intensive wavefront,but a lower thermal impact than a compute intensive wavefront. In otherwords, a memory bound application or wavefront will have lower steadystate temperature when compared to a compute intensive application orwavefront. Thus, if a particular CU is executing a compute intensivewavefront, the temperature of the neighboring CUs gets impacted due tothermal coupling. Therefore, the logic 116 distributes wavefronts toselected CUs to prevent a reduction in the amount of performancewavefronts currently being executed by neighboring CUs or about to bescheduled to the neighboring CUs.

To limit or reduce the thermal impact of the wavefront, thecharacteristic unit 502 determines the characteristic of the wavefront.In some example, the characteristic unit 502 determines thecharacteristic of the wavefront based on analyzing threads within thewavefront. For example, by determining a ratio between a number ofoperations within the wavefront and a number of bits transferred for thewavefront (e.g., the number of bits transferred between secondprocessor, such as GPU 104, and the system memory 120), thecharacteristic unit 502 determines whether the characteristic of thewavefront is a computational intensive, a memory intensive, or mixedintensive. In other words, if the ratio between, the number ofoperations within the wavefront and the number of bits transferred forthe wavefront is large, then the characteristic unit 502 determines thewavefront is computational intensive. If it is small (e.g., close to 0),the characteristic unit 502 determines the wavefront is memoryintensive. If it is in-between, the characteristic unit 502 determinesthe wavefront is mixed.

In some variations, the characteristic unit 502 uses more or less thanthe three different types of wavefronts (e.g., computational, memory,mixed) to categorize the wavefront. In one variation, the characteristicunit 502 uses five or ten classifications (e.g., characteristics) ofwavefronts. For example, if the ratio between a number of operationswithin the wavefront and the number of bits transferred for thewavefront is above a first threshold, then the characteristic unit 502determines the characteristic of the wavefront is a firstcharacteristic. If the ratio between a number of operations within thewavefront and the number of bits transferred for the wavefront is abovea second threshold and below the first threshold, then thecharacteristic unit 502 determines the characteristic of the wavefrontis a second characteristic, and so on.

At step 606, the logic 116 determines thermal information for the GPU104, such as predictive thermal information indicating expected and/orpredicted temperatures associated with the CUs and/or a sub-set of theCUs (e.g., CUs 206, 208, 210, 212). For example, the neighborhoodcircuit determination unit 504 determines expected or predictedtemperature readings of the CUs and/or a subset of the CUs based oninformation 510 indicating locations of the components within the GPU104 and/or other GPUs 106, 108, or 110.

For instance, the neighborhood circuit determination unit 504 obtainsinformation 510 indicating locations (e.g., x, y, and/or z coordinates)of components within the GPU 104 (e.g., intra-chip components) and/orother GPUs 106, 108, 110 (e.g., inter-chip components). Referring toFIG. 2, among other components, the GPU 104 includes non-CU circuitry220, non-CU circuitry 222, and/or the processing units 114. Theinformation 510 indicates locations or the x, y, and/or z coordinates oflocation of the components (e.g., the non-CU circuitry 220, 222) withinthe GPU 104. Additionally, and/or alternatively, the information 510indicates locations of individual CUs within the processing units 114,such as CUs 206, 208, 210, 212. Similarly, in some examples, theneighborhood circuit determination unit 504 also obtains locations ofinter-chip components for the other GPUs 106, 108, and/or 110, whichwill be described below in FIG. 8.

The neighborhood circuit determination unit 504 uses information 510 todetermine expected or predicted temperatures for at least a portion ofthe GPU 104, such as a portion including one or more CUs and/or one ormore sub-sets of CUs 206. For instance, particular components of theGPUs are heat sources and have an expected or predicted temperatureoutput associated with typical operation of the component. For example,if the non-CU circuitry 220 is a memory controller, the neighborhoodcircuit determination unit 504 determines an expected or predictedtemperature of the memory controller during normal operation. Similarly,the neighborhood circuit determination unit 504 determines expected orpredicted temperatures for the CUs, including the portion of CUs and/orother CUs within the GPU 104.

Based on the information 510, the neighborhood circuit determinationunit 504 generates information 516 indicating expected or predictedtemperatures at different locations within the GPU 104. For example,using the x, y, and/or z coordinates of the components within the GPU104 and the expected or predicted temperatures of the components, theneighborhood circuit determination unit 504 generates an expected orpredicted heat map of at least a portion of the GPU 104 (e.g., a portionsurrounding the CUs and/or subset of CUs). The circuit determinationunit 504 provides the predictive thermal information 516 to thegranularity scheduler unit 508 and/or stores the information 516 inmemory, such as cache memory or system memory.

In some variations, the neighborhood determination unit 504 also usesinformation 512 indicating characteristic of the wavefront and theselected CUs to execute the wavefront to determine and/or update thepredictive thermal information 516. In other words, the neighborhooddetermination unit 504 determines where the CUs 206, 208, 210, 212 arelocated (x, y, z coordinates) and the wavefronts and/or characteristicsof the wavefronts being executed by the CUs 206, 208, 210, 212.

For example, as will be explained below, the granularity scheduler unit508 selects one or more CUs (e.g., from the CUs 206, 208, 210, 212) toexecute the wavefront. Further, as mentioned above, the granularityscheduler unit 508 obtains characteristics for the wavefront. After theselection, the neighborhood determination unit 504 obtains information512 indicating the characteristic of the wavefront and/or the selectedCUs to execute the wavefront. As mentioned above, the information 510indicates the location of the CUs 206, 208, 210, 212. Using information510 and 512, the neighborhood determination unit 504 determines thelocations of the CUs 206, 208, 210, 212 and thewavefronts/characteristics of the wavefronts being executed by the CUs206, 208, 210, 212. The characteristics of the wavefronts are associatedwith certain expected or predicted temperatures (e.g., computationalintensive wavefronts have a first temperature, memory intensivewavefronts have a second temperature, and mixed wavefronts have a thirdtemperature). Based on the determination, the neighborhood determinationunit 504 generates and/or updates the predictive thermal information 516indicating expected or predicted temperatures at different locationswithin the GPU 104.

In some examples, after each selection or after a group of selections(e.g., 5 selections), the neighborhood determination unit 504 updatesthe predictive thermal information 516 using the selected CUs 206, 208,210, 212 and the characteristics of the wavefronts. Additionally, and/oralternatively, after execution of the wavefront ends, the neighborhooddetermination unit 504 updates the predictive thermal information 516 toindicate the CUs 206, 208, 210, 212 have finished executing thewavefront. As such, the neighborhood determination unit 504 constantlyupdates the predictive thermal information 516 to indicate the latestpredicted and/or expected temperatures for the GPU 104, including theCUs 206, 208, 210, 212 executing wavefronts with differentcharacteristics.

At step 608, the logic 116 (e.g., the historical and current impactdetermination unit 506) determines thermal information for the CUs, suchas historical thermal information 518 indicating historical or currentthermal temperatures, of at least a portion of the GPU 104. The unit 506provides the historical thermal information 518 to the granularityscheduler unit 508. For example, referring to FIGS. 2 and 3, temperaturesensors 214 provide information 216 indicating current temperatures ofthe CUs 206, 208, 210, 212 and/or temperatures at different locations ofthe processing units 114. Additionally, and/or alternatively, additionaltemperature sensors 214 are included at locations within the GPU 104,such as operatively coupled to the non-CU circuitry 220 and/or 222and/or at various other locations within the GPU 104.

The historical and current impact determination unit 506 (unit 506)obtains information 216 and generates historical thermal information 518using the information 216. In other words, the unit 506 generatesinformation 518 indicating a heat map for a portion of the GPU 104, suchas for a region surrounding a CU or set of CUs, the processing units114, and/or the entire GPU 104. In some examples, the information 216indicates a particular temperature sensor 214 and the correspondingtemperature reading but does not indicate the location of thetemperature sensor 214. As such, the unit 506 obtains information 510indicating locations of components, such as the temperature sensors 214,within the GPU 104. The unit 506 uses the information 510 and 216 togenerate information 518 indicating the heat map for a portion of theGPU 104.

In some examples, information 518 indicates current temperatures andprevious temperatures (e.g., 5 or 10 previous temperatures) of the GPU104. For example, after generating a heat map indicating the currenttemperatures for the GPU 104, the unit 506 stores in memory (e.g., cachememory 118 and/or system memory 120) the current heat map. Then, theunit 506 retrieves the heat maps for previous temperatures of the GPU104 from memory and provides information 518 indicating current andprevious temperatures to the granularity scheduler unit 508.

FIG. 7 shows an exemplary heat map for a GPU, such as GPU 104. Forexample, three heat maps 710, 720, and 730 of a second processor, suchas GPU 104, are shown. The heat maps 710, 720, 730 indicate the currenttemperatures of the GPU 104 and/or the stored (e.g., previous)temperatures of the GPU 104. As shown in the heat maps 710, 720, 730,the unit 506 determines and/or categories portions of the GPU 104 basedon the temperature readings. For example, based on the temperaturereadings, the unit 506 determines whether the portion of the GPU 104 hasa very high temperature reading 702, a high temperature reading 704, amedium temperature reading 706, and/or a low temperature reading 708.The unit 506 provides the information indicating the heat maps 710, 720,730 to the granularity scheduler unit 508.

At step 610, the logic 116 (e.g., the granularity scheduler unit 508)selects a subset (e.g., one or more) of the CUs 206, 208, 210, 212 toprocess the wavefront (e.g., threads) based on the determinedcharacteristic of the wavefront and the temperature information (e.g.,historical thermal information 518 and/or the predictive thermalinformation 516). For example, the granularity scheduler unit 508 (unit508) selects one or more CUs to process or execute the wavefront tominimize the thermal impact on the GPU 104. In other words, the CUs areselected based on characteristics of the wavefront and/or thermalinformation (e.g., proximity of predicted or actual heat sources to CUs)to minimize and/or reduce the cost (e.g., temperature cost) for the CUs206, 208, 210, 212 within the GPU 104.

In some examples, unit 508 uses data representing a function to minimizethe thermal impact (e.g., temperature cost). For example, the unit 508uses the below function to minimize the thermal impact.

$\min {\sum\limits_{i = 1}^{N_{CU}}\; \tau_{i}}$where  τ_(i) = l_(i)w_(i)

For instance, τ_(i) represents the thermal impact of a compute unit(CU), such as compute unit, i, to execute the wavefront. w_(i)represents the thermal impact associated with the characteristics of thewavefront to be scheduled on the CUs (e.g., information 514 indicatingthe characteristic of the wavefront). N_(CU) represents the total numberof compute units within the processing units 114 and/or the GPU 104.l_(i) represents the thermal impact associated with the physicallocation of the CU. In other words, l_(i) represents the proximity towell-known, predicted, or expected heat sources (e.g., components of theGPU) and/or the temperature history of the neighboring CUs and/or theGPU 104 (e.g., the current and/or historical temperature readings of theGPU). In other words, l_(i) represents and/or is based on the predictivethermal information 516 and/or the historical thermal information 518.The unit 508 solves the equation to determine and select the CUs 206,208, 210, 212 that will minimize the thermal impact (e.g., thetemperature cost).

In some instances, the unit 508 solves the equation above using O(n logn) operations where n=N_(cu) (e.g., the number of total compute units).For example, the unit 508 determines two arrays (e.g., a containerobject that holds a fixed number of values of a single type) based onthe equation above. The unit 508 then multiplies a minimum element ofone array with a maximum element of the other array to determine and/orselect the CUs 206, 208, 210, 212 that will minimize the thermal impact.In other words, the unit 508 determines two arrays, one array for l_(i)and another array for w_(i). The unit 508 sorts the arrays (e.g.,ascending for l_(i) and descending for w_(i)) and multiplies the arraystogether to determine/select the CUs.

In some variations, the unit 508 selects a single CU from the CUs 206,208, 210, 212 to execute or process the wavefront. In other variations,the unit 508 selects two or more CUs 206, 208, 210, 212 to execute thewavefront. Additionally, and/or alternatively, the selected CUs are notdirectly adjacent to each other. For example, the unit 508 uses thefunction above and selects idle CUs that are not adjacent, such as oneidle CU from CU 206 and another idle CU from CU 212. Additionally,and/or alternatively, the unit 508 uses the function above to distributethe wavefront across the GPU 104 and/or processing unit 114. Forexample, the unit 508 uses the function to distribute and skip a numberof idle CUs (e.g., every other idle CU or every two idle CUs) to executeor process the wavefront.

In some variations, the unit 508 selects CUs based on their physicallocation on the GPU 104 since some CUs dissipate heat better thanothers. For instance, in some examples, the air flows from right toleft. The CUs 208, 212 in the right side of the GPU 104 havecomparatively lower temperature than the left side 206, 210. Further,CUs on the left most side and CUs in the middle are closer to peripherallogic (e.g., non-CU circuitry) which results in relatively highertemperature. As such, the unit 508 selects CUs based on the ability forCUs to dissipate heat (e.g., right side of GPU 104 for compute intensiveand left side for memory intensive).

At step 612, the logic 116 (e.g., unit 508) provides instructions (e.g.,information 204) to the selected subset (one or more) of CUs 206, 208,210, 212 to execute the wavefront. For example, the logic 116 providesinformation 204 via bus 302 to the different CUs 206, 208, 210, 212 toexecute the wavefront. In some instances, the logic 116 selects CUs toexecute wavefronts at different levels of granularity based on timingconstraints. For example, as described above, the logic 116 selects CUsto wavefronts one by one. In other instances, the logic 116 selects CUsto execute wavefronts in larger batches (e.g., in groups of 5 or 10wavefronts at a time).

FIG. 8 illustrates an example embodiment and/or layout of the GPUs 104,106, 108, 110 within the apparatus 100. For example, GPUs 104, 106, 108,110 are stacked or layered on top of each other within the apparatus100. In other words, the GPUs 104, 106, 108, 110 are in 2.5-D or 3-Dstacks. Each GPU 104, 106, 108, 110 includes temperature sensors 214 andprocessing units 114 with CUs. Further, each GPU 104, 106, 108, 110includes at least one thermal aware optimization logic 116 that providesinstructions to the CUs to execute wavefronts.

In some examples, the logic 116 obtains information, such as information510, indicating locations of inter-chip components of the other GPUs.The logic 116 uses this information 510 to generate and/or update thepredictive thermal information 516 and/or the historical thermalinformation 518. The logic 116 then provides the information 518 and 516to the granularity scheduler unit 508. For example, the logic 116 islocated at a first GPU, such as GPU 108. The logic 116 obtainsinformation 510 indicating locations of components from other GPUs, suchas GPUs layered above (e.g., GPU 106) and/or below (e.g., GPU 110) thefirst GPU. Similar to above, the logic 116 of the GPU 108 usesinformation 510 to generate and/or update predictive thermal information516 indicating expected or predicted temperatures for the GPU 108.

Additionally, and/or alternatively, the logic 116 is in communicationwith components from the other GPUs (e.g., inter-chip components). Assuch, the logic 116 obtains wavefronts being executed by CUs from theother GPUs (e.g., GPU 106 and/or 108). The logic 116 generates and/orupdates the predictive thermal information 516 based on the wavefrontsbeing executed by CUs from the other GPUs. Then, the granularityscheduler unit 508 uses the predictive thermal information 516indicating the wavefronts being executed by CUs from other GPUs toselect CUs to execute the wavefront.

Additionally, and/or alternatively, the logic 116 is in communicationwith temperature sensors 214 from the other GPUs (e.g., GPU 106 and/or108). The logic 116 uses information from the temperature sensors 214from the other GPUs to generate historical thermal information 518indicating current and/or historical temperature information for theother GPUs. Then, the granularity scheduler unit 508 uses the historicalthermal information 518 indicating the temperature information for theother GPUs to select CUs to execute the wavefront.

Among other technical benefits, some examples allow performance of theGPUs to be boosted for comparatively longer periods of time. Forexample, by using the thermal aware optimization logic 116, the CU powercan be boosted by up to 20% till 95° C. is reached (assuming 95C is thecritical temperature limit). The additional power gets translated tobetter performance.

The above detailed description of the present disclosure and theexamples described therein have been presented for the purposes ofillustration and description only and not by limitation. It is thereforecontemplated that the present disclosure covers any and allmodifications, variations or equivalents that fall within the spirit andscope of the basic underlying principles disclosed above and claimedherein.

What is claimed is:
 1. A method for providing workload distribution of aplurality of threads among a plurality of compute units (CUs),comprising: determining a characteristic of the plurality of threads;determining temperature information corresponding to at least a subsetof the plurality of CUs, wherein the temperature information comprises:predictive thermal information indicating expected temperaturescorresponding to at least the subset of the plurality of CUs; andhistorical thermal information indicating current or past thermaltemperatures of at least a portion of the GPU; selecting the subset ofthe plurality of CUs to process the plurality of threads based on thedetermined characteristic and the temperature information; and providinginstructions to the selected subset of the plurality of CUs to executethe plurality of threads.
 2. The method of claim 1, wherein thedetermining the predictive thermal information comprises: determining aplurality of characteristics for a plurality of second threads currentlybeing executed by the plurality of CUs; and generating the predictivethermal information based on the plurality of characteristics for theplurality of second threads currently being executed by the plurality ofCUs.
 3. The method of claim 1, wherein the at least the portion of theGPU comprises at least one non-CU circuitry, and wherein the determiningthe predictive thermal information comprises generating the predictivethermal information based on a location of the at least one non-CUcircuitry.
 4. The method of claim 1, further comprising: obtainingtemperature readings from a plurality of temperature sensors; andgenerating the historical thermal information based on the temperaturereadings from the plurality of temperature sensors.
 5. The method ofclaim 1, wherein the subset of the plurality of CUs is on a secondprocessor, and wherein the method further comprises: determining aplurality of characteristics for a second plurality of threads currentlybeing executed by a second plurality of CUs on a third processor,wherein the second processor and the third processor are stacked on topof each other; and generating the predictive thermal information basedon the plurality of characteristics for the second plurality of threadscurrently being executed by the second plurality of CUs.
 6. The methodof claim 5, wherein the third processor comprises at least one non-CUcircuitry, and wherein the determining the predictive thermalinformation comprises generating the predictive thermal informationbased on a location of the at least one non-CU circuitry.
 7. The methodof claim 6, wherein the third processor comprises a second plurality oftemperature sensors, and wherein the method further comprises: obtainingtemperature readings from the second plurality of temperature sensors;and generating the historical thermal information based on thetemperature readings from the second plurality of temperature sensors.8. The method of claim 1, wherein the determining the characteristic ofthe plurality of threads comprises: determining whether the plurality ofthreads is a memory intensive plurality of threads, a computationalintensive plurality of threads, or a memory and computational intensiveplurality of threads.
 9. The method of claim 8, wherein the selectingthe subset of the plurality of CUs is based on whether the plurality ofthreads is the memory intensive plurality of threads, the computationalintensive plurality of threads, or the memory and computationalintensive plurality of threads.
 10. An apparatus for providing workloaddistribution of a plurality of threads among a plurality of CUs,comprising: a second processor, wherein the second processor comprises:a plurality of compute units (CU) configured to execute the plurality ofthreads; and thermal aware optimization logic, wherein the thermal awareoptimization logic is configured to: determine a characteristic of theplurality of threads; determine temperature information corresponding toat least a subset of the plurality of CUs, wherein the temperatureinformation comprises: predictive thermal information indicatingexpected temperatures corresponding to at least the subset of theplurality of CUs; and historical thermal information indicating currentor past thermal temperatures of at least a portion of the GPU; selectthe subset of the plurality of CUs to process the plurality of threadsbased on the determined characteristic and the temperature information;and provide instructions to the selected subset of the plurality of CUsto execute the plurality of threads.
 11. The apparatus of claim 10,wherein the thermal aware optimization logic is configured to determinethe predictive thermal information by: determining a plurality ofcharacteristics for a plurality of second threads currently beingexecuted by the plurality of CUs; and generating the predictive thermalinformation based on the plurality of characteristics for the pluralityof second threads currently being executed by the plurality of CUs. 12.The apparatus of claim 10, wherein the portion of the GPU comprises atleast one non-CU circuitry, and wherein the thermal aware optimizationlogic is configured to determine the predictive thermal information bygenerating the predictive thermal information based on a location of theat least one non-CU circuitry.
 13. The apparatus of claim 10, whereinthe second processor further comprises a plurality of temperaturesensors configured to provide temperature readings to the thermal awareoptimization logic, and wherein the thermal aware optimization logic isconfigured to: obtain temperature readings from the plurality oftemperature sensors; and generate the historical thermal informationbased on the temperature readings from the plurality of temperaturesensors.
 14. The apparatus of claim 10, further comprising: a thirdprocessor, wherein the second processor and the third processor arestacked on top of each other, wherein the third processor comprises asecond plurality of CUs, and wherein the thermal aware optimizationlogic is configured to determine the predictive thermal information by:determining a plurality of characteristics for a second plurality ofthreads currently being executed by the second plurality of CUs; andgenerating the predictive thermal information based on the plurality ofcharacteristics for the second plurality of threads currently beingexecuted by the second plurality of CUs.
 15. The apparatus of claim 14,wherein the third processor comprises at least one non-CU circuitry, andwherein the thermal aware optimization logic is configured to determinethe predictive thermal information by generating the predictive thermalinformation based on a location of the at least one non-CU circuitry.16. The apparatus of claim 15, wherein the third processor comprises asecond plurality of temperature sensors configured to providetemperature readings to the thermal aware optimization logic, andwherein the thermal aware optimization logic is configured to: obtaintemperature readings from the second plurality of temperature sensors;and generate the historical thermal information based on the temperaturereadings from the second plurality of temperature sensors.
 17. Theapparatus of claim 10, wherein the thermal aware optimization logic isconfigured to determine the characteristic of the plurality of threadsby: determining whether the plurality of threads is a memory intensiveplurality of threads, a computational intensive plurality of threads, ora memory and computational intensive plurality of threads.
 18. Theapparatus of claim 17, wherein the thermal aware optimization logic isconfigured to select the subset of the plurality of CUs based on whetherthe plurality of threads is the memory intensive plurality of threads,the computational intensive plurality of threads, or the memory andcomputational intensive plurality of threads.
 19. An apparatus forproviding workload distribution of a wavefront among a plurality of CUs,comprising: a second processor, wherein the second processor comprises:a plurality of compute units (CU) configured to execute a plurality ofthreads; and thermal aware optimization logic, wherein the thermal awareoptimization logic is configured to: determine a characteristic of theplurality of threads; determine a plurality of characteristics for aplurality of second threads currently being executed by the plurality ofCUs; and generating predictive thermal information based on theplurality of characteristics for the plurality of second threadscurrently being executed by the plurality of CUs; obtain temperaturereadings from a plurality of temperature sensors; generate historicalthermal information based on the temperature readings from the pluralityof temperature sensors; select a subset of the plurality of CUs toprocess the plurality of threads based on the determined characteristic,the predictive thermal information, and the historical thermalinformation; and provide instructions to the selected subset of theplurality of CUs to execute the plurality of threads.
 20. The apparatusof claim 19, further comprising: a third processor, wherein the secondprocessor and the third processor are stacked on top of each other,wherein the third processor comprises a second plurality of CUs, aplurality of second temperature sensors, and wherein the thermal awareoptimization logic is configured to generate the predictive thermalinformation by: determining a second plurality of characteristics for athird plurality of threads currently being executed by the secondplurality of CUs; and generating the predictive thermal informationbased on the second plurality of characteristics for the third pluralityof threads currently being executed by the second plurality of CUs, andwherein the thermal aware optimization logic is configured to generatethe historical thermal information by: obtaining temperature readingsfrom the plurality of second temperature sensors; and generating thehistorical thermal information based on the temperature readings fromthe plurality of second temperature sensors.